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ABSTRACT 



An electronic camera uses a relatively more complex digital 
image processing technique in a still image mode to produce 
high quality still images, and a relatively more simple image 
processing technique in a motion preview mode to produce 
preview images of acceptable quality prior to initiation of 
the still image mode. The more complex digital technique is 
done in software in a general purpose processor section 35, 
while the more simple digital technique is implemented in a 
fixed digital circuit in an application specific integrated 
circuit 27, which also implements timing and control func- 
tions. The motion preview mode uses a shorter image 
readout period than the still mode and further involves 
mapping image sensor pixels into a fewer number of color 
display pixels on a color LCD display 10. The mapping 
further converts color pixel signals from a mosaic array into 
a different color pattern on the color LCD display 10. 

28 Claims, 10 Drawing Sheets 
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ELECTRONIC CAMERA FOR INITIATING In camcorders, the prcx:essiiig for both the still images and 

CAPTURE OF STILL IMAGES V^WLE the motion images is identical. Such processing is normaUy 

PREVIEWING MOTION IMAGES implemented by hardwired analog integrated circuits, 

although camcorders which use digital image processing 

This application is a division of 08/367399 filed Dec. 30, 5 integrated circuits have been produced. Such camcorders 

1994. convert the signal from the CCD sensor into an NTSC 

composite or component formal signal, which is provided to 
FIELD OF THE INVENTION ^ ^j^j^q recording subsystem or a video output jack. The 
The invention pertains to an electronic slill camera for color LCD display includes circuitry to decode the NTSC 
composing and capturing still images, and, more lO composite or component signal back into spatially sub- 
particularly, to an electronic camera having a "motion*' sampled RGB signals to drive the individual RGB pixels on 
mode for previewing a scene and a "still" mode for capturing the LCD sensor. 

a particular image in the scene. In a system oriented toward still photography, and in 

BACKGROUND OF THE INVENTION P,^^^^' ? ^^^1*^ ll^^^i^^ ^ ^^^^^^^ !° 

IS the necessity of generatmg an NTSC format signal in order 

Consumer camcorders which include the capability of to reduce the complexity of the required circuitry. In a totally 

recording analog motion and/or stUl images on 8mm or VHS digital system, that is, both the recording and display chan- 

videotape have been developed by a number of companies. nels are digital, it is further desirable to minimize incom- 

Motion images are recorded in the same manner as in any patibility between the channels. The problem is to achieve 

standard camcorder. These cameras include a single chip 20 these objective in an architecture that minimizes cost and 

charge coupled device (CCD) sensor having a color filter complexity and maximizes user handling. 

array that provides a spatially color-sampled image. To 

record still images, the user pushes a "stiU capture" button SUMMARY OF THE INVENTION 
at the desired instant. The image obtained from the CCD problem is solved according to the invention by a 
sensor is temporarily stored m a digital memory. The unage 25 number of features. In one aspect, the electronic camera is 
IS then read from the memory and recorded onto the video- operable in a still image mode according to a relatively more 
tape. Some camcorders include color liquid crystal displays complex digital image processing technique to produce high 
(LCD), which are also spatially color-sampled devices. ^^^^^^y ^^^^ -^^^^^^^ 3 ^^^-^^ p,^^^^ ^^^.d- 
Sorae are relatively large, for example, ranging from 3 relatively more simple digital image processing 
approximately 2.5" to 4" in diagonal. Such a display is used, 30 technique to produce a preview image of acceptable quality 
instead of a normal eyepiece viewfinder, to aUow the user to p^or 10 initiation of the stiU image mode. Such an archi- 
properly frame the subject and view the images as they are ^^^^^^^ particularly adapted to mapping an array of color 
being recorded. It is also used to view the recorded images ^^^^ p^^^j^ ^ array of color display 
as the videotape is played back, p^^^ls on an LCD display comprising discrete LCD display 
nC. lA shows a typical color LCD display, in which the 35 pixels fewer in number than image sensor pixels. In that 
liquid crystal material is trapped between an upper glass case, a relatively simple digital processing technique corn- 
plate 1 and a lower glass plate 2. The upper plate 1 has a bines sameoolored image pixel signals into a fewer number 
common transparent electrode 3 and an array 4 of color of intermediate pixels that correspond to the arrangement of 
fillers .surrounded by a black mask 5, ITie lower plate 2 (^q color display pixels. 

includes an array 6 of transparent pixel eleclrodes juxla- 40 jhe advantage of the invention is that the two modes can 

posed underneath the array 4 of color filters. Individual pixel ^ relatively low quality "motion" mode and 

elearodes are activated Wathm film transistors CTFD 7^ a much higher quality "still" mode. The motion mode 

are controlled from a video signa I on the source hnes 8 and ^ (>CD sensor arc processed by a hardwired 

a scannmg signal 00 the gate hne 9. The LCD display j^^j 3- j processing circuit that generates low 

includes the iisual polarizer layere (not shown) on the glass 45 resolution, spatially subsampled digital image data which 

plates 1 and 2, such that activation of selected transparent ^.^^^^j ^^ve the relatively low resolution LCD display, 

pixel eleclrodes aUows hght to pass through the correspo^ ^^^^3 complexity and clock frequency of the 

mg color fillere and reflect to the viewer, thereby creaUng a ^^^^^ ^^^^^^ compared to generating an NTSC format 

^ ^^'"'^ i-^^f ^P^'j: ^""^^ ^P"^" ^ signal, as is noraially done in the prior art. The still mode 

2F-Ba)0, manufactured by Seiko-Epson Company, Japan, 50 ^ ^^D sensor is processed by a general 

has about 240 Imes of pixe^ and about 300 pixels per hne, processor (CPU) which executes an image process- 

with an image aspect raUo of 4:3 Such an aspect ratio allows • ^^^^^^^ ^ j„ ^^^^ ^ ^^^^ ^ ^ ,j 

the entire area of the image obtained from the 4:3 aspect dieilal still imaee 

ratio NTSC format CCD sensor to be displayed on the LCD ^ ^ ' 

screen, so that the LCD screen composition will be the same 55 BRIEF DESCRIPTION OF THE DRAWINGS 

as the image that is recorded by the camcorder NTSC format t,, . -n ^ j -u j • 1 . 

' KiTH-^ f . . 1 • The invention will be descnbed in relation to the 

recorder, for later display on an NTSC format television , whereon 

display. Note that because the LCD has only 240 lines of ^ , , . 

pixels, the interlaced NTSC signal is displayed using a ^ ^how the structure and color filter 

"repeat field" technique, where both the odd and even fields 60 P^"^™ ^ ^^^^^ ^^"^^ ^^P^^y (^^D); 

from the NTSC format sensor are displayed using the same P'G. 2 is a block diagram of an electronic camera incor- 

lines of pixels on the LCD. This LCD, like most commer- porating dual modes for composing and capturing a still 

ciaUy avaUable LCDS, has "rectangular" pixels, rather than image according to the invention; 

square pixels, where the distance between pixels in the FIGS. 3A and 3B are diagrams of progressive scan image 

horizontal direction is for example % the distance in the 65 sensors useful with the camera of FIG. 2; 

vertical direaion. The LCD pixels are overlaid with a FIG. 4 is a diagram of the Bayer color filter geometry for 

diagonal RGB stripe pattern as shown in FIG. IB. the sensor used with the camera of FIG. 1; 
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FIG. 5 shows the line timing for the still mode of EPROM memory 42 is used to store the firmware which 

operation; operates the processor 36. The components of the processor 

HGS. 6A and 6B show the line liming for the preview ^5 arc interconnected through a data bus 43, which also 

mode of operation* connects to the timing and control section 27 and to the card 

' < interface 40. 

no. 7 shows Ihe special liae skipping patteni used in the motor^ven zoom lens 22 includes a zoom moior 

preview mode, ^ ^^^^ motor 46, and an aperture motor 48 (all con- 

HG. 8 shows further deuil of the preview mode process- trolled by lens motor drivers 50). The timing and control 

ing circuit shown in FIG. 2; section 27 further includes a control interface 52 conr^ccted 

FIGS. 9A and 9B show further detail of the image iq to the lens motor drivers 50 and to a flash control circuit 53 

statistics processor shown in FIG. 2; via a photosystem interface block 54, which controls the 

FIG. 10 shows one example of still mode image process- operation of the zoom lens 22 and the flash 18. The lens 

iiig; zoom position is controlled by the photosystem interface 

FIG. 11 shows the effect of pixel mapping from a sensor ^.^'f^ ^^^^"^ input from the zoom control 

to an LCD display, each having different aspect ratios; and 15 buttons 14 through a user status and control secuon 55. The 

^ r focusing, exposure control, and white balance is done 

FIG. 12 shows an enhancement to the block diagram of a^^toi^atically, as is typically the case in consumer camcord- 

FIG 2 ID which a different sensor clock frequency is used in ^^^^ ^y computing "image statistics'* in an image 

each of the dual modes. smis^cs processor 60 in the real-time ASIC (timing and 

DETAILED DESCRIPTION OF THE 2° f ^'l^^" ^ previewimages are continuously read 

PREFERRED EMBODIMENT f^f '"^^^^ ^.'^f ^0. The comptited values arc then 

used by a program unplemented m the digital processor 36, 

Ablock diagram of a camera incorporating dual modes of which decides how to adjust the focus motor, aperture, 

processing according to the invention is shown in FIG. 2. analog gain control, and analog white balance controls via 

The camera includes an electronic color display, for 2S the control interface 52 and the photosystems interface 54 on 

example, a color liquid crystal (LCD) display 10 of the type the ASIC timing and control section 27. Although the digital 

shown in FIG. 1 A, and a user control section 12 having a processor 36 and the control interface 52 are shown as being 

number of user control buttons, including zoom buttons 14, within two separate sections, in some implementations the 

a preview button 15 and a capture button 16. To take a still same component could be used to perform both of these 

picture, the user turns on the camera (using a power switch 30 functions (as well as other of the recited functions). Sensor 

(not shown), which may be automatically enabled when the image data is passed to the processor section 35 through a 

user depresses the zoom buttons 14 or the preview button 15, high speed interface 56 in the timing and control section 27. 

or partially depresses the capture button 16). The user The sensor image data is also directed to the color LCD 

composes the picture by depressing the "zoom in" or "zoom display 10 through a preview mode processing circuit 58. 

out" buttons 14, and by adjusting the position of the camera, 35 The timing and control section (ASIC) 27 is operable in 

whileobservinglhedisplay image. When the user is satisfied two modes, a relatively low quahty "motion" mode and a 

with the composition on the color LCD display 10, the user much higher quality "still" mode. In the motion mode, 

depresses the capture button 16. The camera then capaires a images from the sensor 20 are processed by the preview 

single still image, firing a flash 18 if necessary when the mode processing circuit 58; in the stiU mode, images from 

ambient illumination level is low. The still image is focused 40 the sensor 20 are processed in the processor 35. The pro- 

upon an image sensor 20 by a motor driven zoom lens 22. cessor 35 is a software driven digital processing system that 

The intensity of the image light upon the sensor 20 is is slower than the ASIC 27. The preview mode processing 

regulated by a motor-driven, variable, mechanical aperture circuit 58 is a hardwired digital signal processing circuit 

24, while exposure time is regulated electronically by appro- (part of the ASIC 27) that generates low resolution, spatiaUy 

priate clocking of the sensor 20. The still image from the 45 subsampled digital image data which can directly drive the 

image sensor 20 is processed and digitally stored on a relatively low resolution color LCD display 10. This reduces 

removable memory card 26. the complexity and clock frequency of the required circuitry. 

Control of the sensor is provided by a timing and control compared to generating an NTSC format signal, as is 

section 27, which is an application specific integrated circuit normally done in the prior art. The preview mode processing 

(ASIC) with processing and timing functions, for both 50 circuit includes a pixel remapper 62 for mapping the greater 

capture and preview operating modes. For instance, the number of image pixels from the sensor 20 into the lesser 

liming and control section 27 includes a sensor timing number of display pixels (i.e., corresponding to the array 6 

circuit 28 for controlling the image sensor functions. The of transparent pixel electrodes in FIG. 1) in the color LCD 

sensor liming circuit 28 provides the signals to enable sensor display 10. The color saturation of the remapped pixels is 

drivers 30, which provide horizontal clocks (HI, H2), ver- 55 then adjusted in a color adjustment circuit 63 and its output 

tical clocks (VI, V2), as well as a signal FDG for activating is applied to a multiplexer 64. The multiplexer 64 selects 

a drain structure on the sensor 20. The output of the image image data either from the preview mode processing circuit 

sensor 20 is amplified and processed in an analog gain and 58, producing a preview image, or from the high speed 

sampling (correlated double sampling (CDS)) circuit 32, and interface 56, which allows for suitably preprocessed viewing 

converted to digital form in A/D converter stage 34. The A/D 60 of stored images. 

output signal is provided to a processor section 35, which In this camera, the image processing used to create ihe 

includes a digital processor 36 for temporarily storing the preview mode is done in the timing and control ASIC 27, 

still images in a DRAM memory 38. The digital processor since the processing must be done rapidly. About 60 images 

36 then performs image processing on the still images, and per second are processed in preview mode. However, since 

finally stores the processed images 00 the removable 65 the image quality of the displayed image is limited by the 

memory card 26 via a memory card interface circuit 40, resolution and color gamut of the LCD screen of the LCD 

which may use the PCMCIA 2.0 standard interface. An color display 10, there is no need for elaborate image 
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processing. Therefore, simple "preview mode" image pro- 
cessing is performed in a fixed digital circuit embedded in 
the preview mode processing circuit 58 (which is part of the 
ASIC). The quality requirements for the still mode are much 
greater, since these images will be downloaded to a 
computer, and may be displayed on a high resolution CRT 
display, or printed on a high quality thermal printer. 
Therefore, the digital image processing must be more elabo- 
rate. By using the digital processor 36 to implement software 
procedures stored in the firmware memory 42, complex 
procedures can be implemented. These procedures can take 
several seconds to complete, since real-time operation is not 
required. Use of firmware-stored software allows the still 
mode image processing to be upgraded without requiring a 
new ASIC design. In effect, what happens is thai a relatively 
less complex digital image processing technique is used in 
the motion preview mode, but at a higher data rate, and a 
relatively more complex digital image processing technique 
is used in the still mode, but at a slower data rate. 

Since the update rate, that is, the number of images that 
need to be supplied per unit time, is different for the still 
mode than for the motion mode, it is beneficial to use 
different clock frequencies for the different modes of opera- 
tion. For example, as shown in FIG. 12, a system oscillator 
100 produces a 12 mHz clock frequency for use in the 
motion mode to obtain more updates/second (e.g., 60 images 
per second), while a divider 102 divides by two to provide 
a 6 mHz clock frequency for the still mode. The lower 
frequency allows more time to accurately position the clamp 
and sample pulses so as to avoid CCD output signal tran- 
sitions. This increases noise immunity in the still mode. A 
multiplexer 104 is enabled by the control interface 52 to 
determine which clock frequency is applied to the sensor 
timing circuit 28. Though not specifically shown, the 
changed timing is also communicated to the A/D stage 34 
and other timing and control circuits. 

The sensor 20 is a progressive scan interb'ne image sensor 
having a noninterlaced architecture, as shown in more detail 
in FIG. 3A. ITie sensor comprises a two-dimensional array 
of pholosites 66, e.g. photodiodes, arranged in rows and 
columns of image pixels, a plurality of vertical registers 68 
adjacent photosite columns for transferring rows of image 
pixel charge from the photosites 66 to a horizontal register 
70 for readout responsive to clock signals from the sensor 
drivers 30, and a charge drain (specifically, a fast dump 
structure 72) interposed between the output of the vertical 
registers 68 and the horizontal register 70 for eliminating 
complete rows of image pixels at a time from the image 
sensor 20. A preferred image sensor is the Kodak model 
CCD KAI-0400CM image sensor, which has approximately 
512 active lines with approximately 768 active pixels per 
line and an image aspect ratio of 3:2. lliis sensor is 
described in a Performance Specification document avail- 
able from Eastman Kodak Company, Rochester, N.Y. Each 
pixel is 9 microns "square", since both the vertical and 
horizontal distances between the centers of adjacent pixels 
are 9 microns. The 3:2 image aspect ratio of the CCD sensor, 
although wider than the 4:3 aspect ratio of the display, is 
considered to be a preferred aspect ratio for still 
photography, in that the standard 35 mm film format, and 
standard 4R (4''x6'') prints also have a 3:2 image aspect 
ratio. The sensor uses a color filter array pattern known as 
the "Bayer checkerboard" pattern, described in U.S. Pat. No. 
3,971,065, which is shown in FIG. 4. Such a color filter array 
is characterized by a mosaic pattern in which the filter colors 
alternate in both line and column directions. In the normal 
operating mode, all of the image pixels on the sensor are 
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transferred as color image pixels to the horizontal register 
70, which delivers a stream of color pixel signals to the 
analog gain and CDS circuit 32 (see FIG. 2). The color pixel 
signals are subsequendy converted to digital pixel signals in 

5 the A/D converter 34. 

The sensor 20 uses a progressive scan readout method, 
which allows the entire image to be read out in a single scan. 
The accumulated or integrated charge for the photodiodes 
comprising the pixels 66 is transported from the photosites 

JO to light protected vertical (parallel) registers 68 by applying 
a large positive voltage to the phase-one vertical clock (VI). 
'Iliis reads out every row, or Line, into the vertical registers 
68. The image pixel charge is then U-ansporled from the 
vertical registers 68 to the horizontal register 70 by two- 

35 phase clocking of the vertical clocks (VI, V2). Between the 
vertical and horizontal registers is the fast dump strucmre 
72, which is further described in the Performance Specifi- 
cation document for the KAI-0400CM sensor. By setting a 
suitable positive potential on a fast dump gate line FDG, 

20 charge from the row of pixel values currently adjacent to the 
fast dump structure 72 is transferred from the CCD directly 
into the sensor substrate 74 rather than to the horizontal 
register 70. This dump, or line clear, is accomplished during 
the vertical-to-horizontal transfer time. When properly con- 

25 trolled by the sensor timing circuit 28, the fast dump 
structure 72 allows lines of charge to be eliminated. 

The timing and control section 27 operates the electronic 
camera shown in FIG. 2 in the two aforementioned modes, 
including a first "still" mode wherein all rows of image pixel 

30 charge corresponding to each line are progressively read out 
through the horizontal register 70 during a single scan, and 
a second "motion" mode wherein some of the rows of image 
pixel charge corresponding to some lines are eliminated 
through the fast dump structure 72 prior to readout. As 

35 applied to the embodiment of FIG. 2, the first mode corre- 
sponds to a high quality still imaging mode while the second 
mode corresponds to a special "line skipping" mode for 
driving the color LCD display 10. In the second mode, the 
liming and control section 27 controls the fast dump struc- 

40 ture 72 to eliminate two or more consecutive lines of image 
charge from the image sensor 20 for every one or more lines 
of image charge that are transferred to the horizontal register 
70 for readout, thereby generating a pattern of lines (shown 
in FIG. 7) suitable for driving the LCD display in a "repeal 

45 field" mode. An appropriate video signal which displays the 
entire 3:2 aspect ratio sensor image on the 4:3 aspect ratio 
LCD, without introducing geometric distortion, is generated 
by alternately eliminating two or four consecutive lines of 
image charge for every pair of lines of image charge that are 

50 transferred to the horizontal register 70. 

The sensor timing circuit 28 is controlled by the control 
interface 52 to provide the clock signals VI, V2, HI, H2, and 
the gate signal FDG according to the two modes of opera- 
lion. The timing signals for the first mode are shown in RG. 

55 5; those for the second mode are shown in FIGS. 6a and 66. 
The two-phase cycling of signals VI and V2 control the 
transfer of Hues of image pixel charge from the vertical 
registers 68 to the horizontal register 70. The two-phase 
cycling of signals HI and H2 conUol the transfer of a sU-eam 

60 of color pixel signals from the horizontal register 70 to 
subsequent circuits in the camera. The level of the signal 
FDG determines whether the image charge is dumped to the 
substrate 74 or transferred to the horizontal register 70. 
When the sensor 20 is clocked using the first timing mode 

65 shown in FIG. S for all lines of the sensor, all lines of the 
sensor are clocked out, one after the other, through the 
horizontal register 70, processed in subsequent camera 
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circuitry, and stored in the removable memory 26. This The processing cx)mplexiiy of the camera of FIG. 2 is 

timing mode provides a high quality progressive scan stili considerably simplified by directly mapping the RGB sensor 

image, but may take Vao second or longer to read out the still pixels 66 to the RGB pixels of the display 10. The easiest 

image. Such timing, however, is acceptable for still mode way to do this for the image sensor 20 (512 linesx768 pixels 

usage, and, as mentioned before, does not require unusually 5 and 3:2 aspect ratio) is to have ao LCD display with 

high speed components and, indeed, may benefit from a 512x768 pixels and the same aspect ratio and color filter 

lower speed lock. pattern. However, this would be a custom, costly LCD. 

To provide an image to the color LCD display 10, a lower Instead, LCDs have fewer display pixels than image pixels 

resolution image is suitable, but the update rate must be on the image sensor, normally have a 4:3 image aspect ratio, 

suflScient to provide good motion rendition and eliminate jq and use the diagonally striped RGB pattern shown in FIG. 

display flicker. Moreover, the sensor 20 includes the afore- IB. In this discussion, an LCD pixel array of about 240 

mentioned array of color filters arranged in a particular linesx312 pixels per line is assumed, 

mosaic color pattern (e.g., the checkerboard Bayer pattern of Therefore, the sensor pixels are processed in a "pixel 

FIG. 3), and the lines of image charge that are transferred to mapping circuit", such as the LCD pixel remapper 62. A 

the horizontal register 70 should preserve that particular ^5 block diagram of this circuit is shown in RG. 8. Note that 

color pattern in the pattern of lines that are generated for the there are 768/2-384 green or red/blue pixels per line on the 

line-skipping readout. To provide this kind of image, the sensor (see FIG. 4). There are about 300/3=100 green, red, 

sensor is read out in the second mode as shown in FIG. 7, or blue pixels per line on the LCD (see FIG. IB). Thus, there 

using the timing shown in FIGS. 6A and 6B. As shown in are approximately Va as many LCD pixels per line (per color) 

FIG. 6A, the first two lines (1 and 2) are read out as in the 20 as there are sensor pixels per line. Therefore, the basic plan 

normal mode. These provide a green-red and a blue-green is to combine same-colored image pixels into a fewer 

line. The next two lines (3 and 4) are eliminated by turning number of intermediate, combination pixel signals that are 

on the fast dump structure 72 during the time that these lines then mapped into the color display pixels. For instance, a 

are transferred past the fast dump structure 72. Next, as simple "pixel mapping" circuit maps four green sensor pixel 

shown in RG. 6B, lines 5 and 6 are read out normally, and 25 signals into one green LCD pixel for one line by summing 

then lines 7, 8, 9, and 10 are eliminated. Next, the FIG. 6A two green sensor values, spaced apart by 4 CCD pixel 

timing is used to read out lines 11 and 12, while eliminating positions, in the green pixel summer 76 and dividing by two 

lines 13 and 14, and then the FIG. 6B timing is used to read via bit-shift wiring. The necessary delay is provided by the 

out lines 15 and 16, while eliminating lines 17-20. This registers 82 clocked at one fourth the pixel rate, further 

process proceeds for the entire image readout period, during 3Q delayed by one pixel clock. It also maps four red sensor 

which 102 pairs of lines are read out, and 154 pairs of lines pixels into one LCD pixel in the same manner (using the 

are eliminated. red/blue summer 78), and also stores this value in a 100 pixel 

This special "line skipping" readout mode, as shown in FIFO 80. The FIFO 80 compensates for the fact that the 

FIG. 7, allows the sensor 3:2 aspect ratio image to be fully sensor has line sequential red and blue pixels, by supplying 

displayed on a 4:3 aspect ratio LCD without "geometric 35 blue pixels on the red sensor lines, and red pixels on the blue 

distortion", that is, without stretching the image vertically, sensor lines. Four pixel delays are provided by the registers 

and without cropping off the horizontal edges of the image 82' clocked at one fourth pixel rate (CLK/4). The mapping 

from the image sensor. This allows the LCD to property process is basically, therefore, a process which, in its sim- 

show the entire 3:2 aspect ratio image captured by the plest form, involves averaging of signals to produce a 

sensor, so that an image can be properly composed. 40 smaller number of output color pixels than input color 

As the 512 lines of the CCD imager are read out using the pixels. (The CFA interpolation algorithm discussed in ref- 

special "line skipping" mode, they are displayed using only erence to FIG. 10, on the other hand, produces a larger 

204 out of the approximately 240 LCD lines of pixels. The number of "output" color pixels than input color pixels.) 

remaining approximately 36 lines can cither be masked Alternate groups of 2 or 4 lines of sensor values are 

behind a bezel, so that they are not visible, or preferably may 45 discarded during preview mode by using the fast dump gate 

be used to display status information, such as the lime-of- shown in FIG. 3A, as described in connection with the "line 

day, image number, or a "push-button menu" for the user skipping** mode. This allows the sensor readout time to be 

buttons. FIG. 11 shows a useful application of such conver- decreased by more than during the preview mode, 

sions. A sensor having a 3:2 aspect ratio is shown mapped Another feature of the design is that by removing the 

into an image area 90 of an LCD display screen 92 having 50 NTSC rate driving circuitry from the color LCD display 10, 

a 4:3 aspect ratio. A proportional remapping leaves a status the active matrix LCD can be updated at a slower frequency 

area 94 available for other purposes, specifically to show than is normally used. This reduces the cost and power 

text indicating the function of a set of reconfigurable control consumption of the LCD driver circuits (not shown). For 

buttons 96 in the control section 12. The function of the example, the LCD can be updated at 30 Hz (provided the 

buttons is specified by the user staUis and control section 55 55 LCD active matrix display is designed so as not to exhibit 

(FIG. 2). This status information graphics data can be noticeable flicker at this update rate), instead of 60 Hz. 

supplied by the digital processor to the LCD 10 via high Once the LCD pixel values are calculated, the LCD color 

speed interface 56, when the MUX 64 is controlled so as to adjust circuit 63 increases the color saturation of the image 

use the digital data from interface 56, rather than from by forming R-G and B-G color difference signals, and 

circuit 58, for supplying data to the final approximately 32 60 adding or subtracting a fraction of these signals in an array 

lines of the display 10. 84 of adders and subtractors from the original RGB signals 

The "line skipping" readout causes some minor vertical in order to increase the color saturation of the displayed 

sampling artifacts, but these are not noticeable in the small image. This circuit performs a similar function as a 3x3 

LCD displays. The pixels output for the sensor 20 in line color matrix, but uses less hardware and provides less color 

skipping mode continue to have the Bayer-type color filter 65 accuracy. ll]e color accuracy is not critical for the LCD 

repeating pattern, so that they can be processed using display, however, due to the limited color reproduction 

processing techniques designed for the Bayer pattern. quality of such displays. The color reproduction of the still 
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image is much more importani, and is done with a more pixel pattern, without using a line delay. Since two CCD 

complicated and precise color correction method with the lines are read out in parallel, for each LCD line, fewer lines 

stored firmware in EPROM 42, are eliminated via the fast dump gale than for the single- 

RGS. 9A and 9B show the processing in the image register sensor shown in FIG. 3A. 

statistics processor 60, which computes real-time values for 5 In particular, if the dual register CCD in FIG, 3B had 

both still and motion image capture, all during the preview 512x768 square pixels with a 3:2 aspect ratio, and the LCD 

mode. These values include 24 R, G, and B averages used to had approximately 240 display lines and a 4:3 aspect ratio, 

set white balance and exposure, and 6 average high fre- CCD readout would involve fast dumping one pair of 

quency 'detail values used to set focus. The 24 RGB ^^^^ fo, ^^^^ p^^^ i^^^^ out from the CCD 

averages for white balance are calculated for a group of 4x6 ^^^^ j^^, ^^^^^^ procedure for the horizontal reg- 

rectangular image regions, for each of R, G, and B; a block ^^^^ j^en be varied depending on the mode of opera- 

diagram of the calculation m one color is shown in RG. 9A. ^oth registers are used for the motion imaging mode 

The 6 average "detail" values for focusing are calculated for ^^^^^^^ ^ ^^e sliU imaging mode, 

green pixels only by accumulating the absolute value of the Furthermore, although the Bayer paltem was described, 

differences between nearby green values; a block diagram of 35 ^j^er mosaictype filter patlcrns could be used to advantage, 

this calculation is shoNvn m HG. 9B These values are ^xnmpXc. complementary patterns involving cyan, 

computed for each preview mi age and downloaded to the ^^^^^^^^ ^^^^^ gu^^ The processing for the 

processor 36 Hie processor 36 implements a firmware LCD pixel remapper 62 and the LCD color adjust circuit 63 

stored procedure which determines the optimum exposure ^^^^ ^e accordingly modified to account for the different 

parameters (exposure time, ^stop, and analog gain), while 20 color arrangement, 
balance settings, and lens focus setting. For the still mode, 

the processor 36 also decides, based on the last preview PARTS LIST 

images, whether to fire the flash, and determines the opti- | upper plate 

mum exposure parameters, white balance settings, and lens 2 lower plate 

focus setting for the still mode. 35 3 common transparent electrode 

Once the still image is captured, the digital processor 36 4 array of color filters 

implements the stored firmware procedures to process and 5 black mask 

store the still image. FIG. 10 shows a diagram of one 5 array of transparent pixel electrodes 

possible still image processing method. The CFA interpola- 7 thin film transistors 

tion diagram may include the green interpolation method 30 g source lines 

described in U.S. patent application Ser. No. 085,520, filed 9 gate line 

Jun, 30, 1993, in the name of the same assignee as the color LCD display 

present application, and the chrominance interpolation 12 control section 

method described in U.S. Pat, No. 4,642,678, both of which 14 zoom buttons 

are incorporated herein by reference. The color matrix, tone 35 15 preview button 

correction, and edge enhancement steps may be similar to 15 capture button 

those described in U.S. Pat. No. 5,189,51 1, also incorporated ig flash 

herein by reference. The image compression method may be 20 progressive scan interline 

the JPEG standard compression technique. 22 zoom lens 

The foregoing description envisions taking a single still 40 24 mechanical aperture 

picture following the motion preview mode. The camera can 26 memory card 

also optionally capture "bursts" of high quality still images 27 timing and control section 

into the DRAM memory during the ''still" mode, which are 28 sensor timing circuit 

then processed as shown in FIG. 10. Owing to the flash 30 sensor drivers 

recharge time and other limitations of the "burst" mode, the 45 32 analog gain and CDS 

"burst" mode could utilize difierent exposure parameters 34 A/D converter 

(exposure time, aperture, analog gain, flash, and digital 36 digital image processor 

processing) than either the motion or the single still mode. 38 DRAM memory 

The invention has been described with reference to a 40 card interface 

preferred embodiment. However, it will be appreciated that 50 42 EPROM memory 

variations and modifications can be effected by a person of 44 zoom motor 

ordinary skill in the art without departing from the scope of 46 focus motor 

the invention. For instance, FIG. 3B shows a progressive 48 variable aperture 

scan sensor with two readout registers 86 and 88 (which 50 lens motor drivers 

corresponds to the Performance Specification document for 55 52 control interface 

the Kodak KA1-031CM image sensor; the preferred embodi- 53 flash control circuit 

meot of FIG, 3A simply uses but one register). The purpose 54 photosystems interface 

is to eliminate the FIFO line delay 80 in the LCD pixel 55 user status and control section 

remapper 62. The pairs of lines read out by the registers 56 high speed interface 

include both a green/red line and a blue/green line. 60 58 preview mode processing circuit 

Therefore, by adding an analog multiplexer between the 60 image statistics processor 

outputs of the two channels and the analog gain and CDS 62 pixel remapper 

block 32, which is switched at the sensor horizontal clock 63 color adjustment circuit 

rate, it is possible to obtain a GRBG sequence of sensor data 64 multiplexer 

values at the output of the A/D stage 34. The LCD pixel 65 66 pixel 

remapper 62 can then be designed to map from the CCD 68 vertical readout register 

sensor color pixel pattern to the required RGBRGB LCD 70 horizontal register 
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72 fast dump structure 
74 sensor substrate 
76 green pixel summer 
78 red/blue summer 
SOfifo 

82 two pixel delay registers 
84 array 

86 first readout register 
88 second readout register 
90 image area 
92 LCD display screen 
94 status area 
96 reconfigurable buttons 
100 oscillator 
102 divider 
104 multiplexer 
What is claimed is: 

1. An electronic still camera for initiating capture of a still 
image while previewing motion images on a display, com- 
prising: 

(a) an image sensor having a two-dimensional array of 
pbotosites covered by a mosaic pattern of color filters 
including at least three dififerent colors for capturing 
images of a scene, each captured image having a first 
number of color pixel values provided in a first color 
pattern; 

(b) motion processing means for generating from the 
captured images, a second number of color pixel values 
provided in a second color pattern having at least three 
different colors and representative of a series of motion 
images to be previewed, the second number of color 
pixel values being less than the first number of color 
pixel values, and the second color pattern being differ- 
ent from the first color pattern; 

(c) a color display for presenting at least some of the 
motion images of the series of motion images corre- 
sponding to the captured images of the scene, the color 
display having an arrangement of color display pixels 
including at least three different colors in a pattern 
different from the first color pattern; 

(d) a capture button for initiating capture of a still image 
while previewing the motion images presented on the 
color display; 

(e) still processing means for generating a third number of 
color pixel values including at least three different 45 
colors representative of a processed captured still 

image; and 

(f) a digital memory for storing the processed captured 
still image. 

2. The electronic still camera of claim 1 further including 50 
a multiplexer having first and second inputs and an output, 
wherein the first input is coupled to the first processing 
means, the second input is coupled to the second processing 
means, and the output is coupled to the display so that a 
captured still image can be viewed on the display. 55 

3. The electronic still camera of claim 1 wherein the first 
processing means and the second processing means are 
integrated into a single integrated circuit. 

4. The electronic still camera of claim 1 wherein the 
motion processing means and the still processing means 
produce the second number of pixels and the third number 
of pixels, respectively, prior to transferring the image pixels 
to the display and the digital memory, respectively. 

5. The electronic still camera of claim 1 wherein the 
second number of pixel values representative of the series of 65 
motion images are transferred to the display in a digital 
format. 



6. The electronic still camera of claim 1 wherein the 
second number of pixel vahies representative of the series of 
motion images are mapped into a form suitable for display. 

7. The electronic still camera of claim 6 wherein after the 
second number of pixel vahies representative of the series of 
motion images arc mapped into a form suitable for display, 
the signals representing the captured images are converted 
back into an analog form. 

8. The electronic still camera of claim 1 further including 
a firmware memory, a processor, and an application specific 
integrated circuit, and wherein the first processing means is 
provided in the application specific integrated circuit, and 
the second processing means is provided using the processor 
which implements software procedures stored in the firm- 
ware memory. 

9. The electronic still camera of claim 1 wherein the 
image sensor includes a color filter array to produce color 
image data, the second processing means processes the color 
image data and performs JPEG image compression on the 
captured still image, and the JPEG compressed still image is 

20 stored in the digital memory. 

10. The electronic still camera of claim 9 wherein the 
digital memory is a removable memory card. 

11. The electronic still camera of claim 1 wherein the fiist 
color pattern includes substantially one-half green color 
filters. 

12. The electronic still camera of claim 11 wherein the 
color di^lay pixels are arranged in a pattern having sub- 
stantially one-third green color filters. 

13. The electronic still camera of claim 1 wherein the first 
color pattern is a Bayer checkerboard pattern. 

14. The electronic still camera of claim 1 wherein the 
distance between adjacent image sensor photosites in the 
horizontal direction are substantially equal to the distance 
between the adjacent image sensor photosites in the vertical 
direction, and wherein the distance between the adjacent 
color display pixels in the horizontal direction is different 
than the distance between the adjacent color display pixels 
in the vertical direction. 

15. An electronic still camera for initiating capture of a 
still image while previewing motion images on a display, 
comprising: 

(a) an image sensor having a two-dimensional array of 
photosites covered by a mosaic pattern of color filters 
including at least three different colors for capturing 
images of a scene, each captured image having a first 
number of color pixel values provided in a first color 
pattern; 

(b) a motion processor for generating from the captured 
images, a second number of color pixel values provided 
in a second color pattern having at least three different 
colors and representative of a series of motion images 
to be previewed, the second number of color pixel 
values being less than the first number of color pixel 
values, and the second color pattern being different 
from the first color pattern; 

(c) a color display for presenting at least some of the 
motion images of the scries of motion images corre- 
sponding to the captured images of the scene, the color 
display having an arrangement of color display pixels 
including at least three different colors in a pattern 
different from the fiist color pattern; 

(d) a capture button for initiating capture of a still image 
while previewing the motion images presented on the 
color display; 

(e) a still processor for generating a third number of color 
pixel values including at least three different colors 
representative of a captured still image; and 
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(f) a digital memory for storing the processed captured 

still image. 

16. The electronic still camera of claim 15 further includ- 
ing a multiplexer having first and second inputs and an 
output, wherein the first input is coupled to the first 
processor, the second input is coupled to the second 
processor, and the output is coupled to the display so that a 
captured still image can be viewed on the display. 

17. The electronic still camera of claim 15 wherein the 



first processor and the second processor are integrated into lo the digital memory. 



grated circuit, and wherein the first processor is provided in 
the application specific integrated circuit, and the second 
processor implements software procedures stored in the 
firmware memory. 

23. The electronic still camera of claim 15 wherein the 
image sensor includes a color filter array to produce color 
image data, the second processor processes the color image 
data and performs JPEG image compression on the captured 
still image, and the JPEG compressed still image is stored in 



a single integrated circuit. 

18. The electronic still camera of claim 15 wherein the 
motion processor and the still processor produce the second 
number of pixels and the third number of pixels, 
respectively, prior to transferring the image pixels to the 
display and the digital memory, respectively. 

19. The electronic still camera of claim 15 wherein the 
second number of pixel values representative of the series of 
motion images arc transferred to the display in a digital 
format. 

20. The electronic still camera of claim 15 wherein the 
second number of pixel values representative of the series of 
motion images are mapped into a form suitable for display. 

21. The electronic still camera of claim 20 wherein after 



24. The electronic still camera of claim 23 wherein the 
digital memory is a removable memory card. 

25. The electronic still camera of claim 15 wherein the 
first color pattern includes substantially one-half green color 

15 filters. 

26. The electronic still camera of claim 25 wherein the 
color display pixels are arranged in a pattern having sub- 
stantially one-third green color filters. 

27. The electronic still camera of claim 15 wherein the 
20 first color pattern is a Bayer checkerboard pattern. 

28. The electronic still camera of claim 15 wherein the 
distance between adjacent image sensor photosites in the 
horizontal direction are substantially equal to the distance 
between the adjacent image sensor photosites in the vertical 



the second number of pixel values representative of the 25 direction, and wherein the distance between the adjacent 



series of motion images are mapped into a form suitable for 
display, the signals representing the captured images are 
converted back into an analog form. 

22. The electronic still camera of claim 15 further includ- 
ing a firmware memory, and an application specific inte- 



color display pixels in the horizontal direction is different 
than the distance between the adjacent color display pixels 
in the vertical direction. 
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